The growth of distributed computing, mobile applications, social media applications, and big data applications (i.e., the collection of large amounts of data or complex data) has led to an increase in object based storage which generally allows for better scalability than file based storage. In many cases, the data objects may never be accessed again, but they are expected to remain accessible if needed. Examples of such data objects can include photos, movies, e-commerce data, or archival data stored across a network as in cloud storage.
In addition, these applications have led to the need for larger storage capacities to retain data. In response, data is often stored in a cluster or group of Data Storage Devices (DSDs) that each includes its own storage media such as rotating magnetic disks or solid-state memories such as flash. In an object based cluster of DSDs such as in Ceph, Hadoop, Swarm, or OpenStack clusters, each DSD is generally treated as a separate Object Storage Device (OSD) that is either available or unavailable for storing data.